Dirac's equation and the nature of quantum field theory

This paper re-examines the key aspects of Dirac’s derivation of his relativistic equation for the electron in order advance our understanding of the nature of quantum field theory. Dirac’s derivation, the paper argues, follows the key principles behind Heisenberg’s discovery of quantum mechanics, which, the paper also argues, transformed the nature of both theoretical and experimental physics vis-à-vis classical physics and relativity. However, the limit theory (a crucial consideration for both Dirac and Heisenberg) in the case of Dirac’s theory was quantum mechanics, specifically, Schrödinger’s equation, while in the case of quantum mechanics, in Heisenberg’s version, the limit theory was classical mechanics. Dirac had to find a new equation, Dirac’s equation, along with a new type of quantum variables, while Heisenberg, to find new theory, was able to use the equations of classical physics, applied to different, quantum-mechanical variables. In this respect, Dirac’s task was more similar to that of Schrödinger in his work on his version of quantum mechanics. Dirac’s equation reflects a more complex character of quantum electrodynamics or quantum field theory in general and of the corresponding (high-energy) experimental quantum physics vis-à-vis that of quantum mechanics and the (low-energy) experimental quantum physics. The final section examines this greater complexity and its implications for fundamental physics. PACS numbers: 01.65.+g, 01.70.+w, 03.65. w, 03.65.Ud, 03.65.TA, 03.70.+k 1. The mathematical correspondence principle, quantum variables and probability Heisenberg’s thinking leading him to his discovery of quantum mechanics was defined by three key elements, clearly apparent in his paper announcing his discovery (Heisenberg 1925). The same elements, I argue here, also defined Dirac’s work on quantum electrodynamics, most especially his discovery of his relativistic equation for the electron, my primary concern in this paper. Dirac’s derivation of the equation was significantly influenced by Heisenberg’s paper. As is well known, Dirac read the paper very carefully in 1925, and it inspired his 1925–6 work on quantum mechanics and quantum electrodynamics, from which, especially his transformation theory (independently discovered by Jordan), his work on his equation grew. It is true that Dirac’s work on his equation was significantly indebted to other developments in quantum mechanics, most especially Schrödinger’s equation, the transformation theory, and Pauli’s spin theory. Nevertheless, Heisenberg’s thinking in his 1925 paper, and specifically the three key points in question, exerted a profound influence on Dirac’s work on his equation. These points are as follows: (1) The mathematical correspondence principle. Stemming from Bohr’s correspondence principle, this principle states that one should maintain the consistency, ‘correspondence,’ between quantum mechanics and classical physics. More specifically, in the region, such as for large quantum numbers for electrons in atoms (when electrons are far away from the nucleus), where one could use classical physics in dealing with quantum processes, the predictions of classical and quantum theory should coincide. The correspondence principle, used more heuristically and ad hoc by Bohr and others before quantum mechanics, was also given a more rigorous, mathematical form by Heisenberg. The mathematical correspondence principle (here extended to quantum electrodynamics and other forms of quantum field theory) requires recovering the equations and variables of the old theory, classical mechanics in the case of quantum 0031-8949/12/014010+09$33.00 1 © 2012 The Royal Swedish Academy of Sciences Printed in the UK Phys. Scr. T151 (2012) 014010 A Plotnitsky mechanics and quantum mechanics in the case of quantum field theory, in the limit region where the old theory could be used. Dirac was aware of this version of the correspondence principle from his first paper on quantum mechanics on. As he said there: ‘The correspondence between the quantum and classical theories lies not so much in the limiting agreement when h ) 0 as in the fact that the mathematical operations on the two theories obey in many cases the same [formal] laws’ (Dirac 1925, p 315). Rather than becoming obsolete after quantum mechanics, as has been sometimes argued, the principle, in this mathematical form, has continued to play a major role in the development of quantum theory. It still does, for example, in string and brane theories, where the corresponding limit theory is quantum field theory. (2) The introduction of the new type variables. Arguably most centrally, both discoveries, that of Heisenberg and that of Dirac, were characterized by the introduction of the new types of variables. (QM) In the case of quantum mechanics, these were matrix variables with complex coefficients, essentially operators in Hilbert spaces over complex numbers (apparently, unavoidable in quantum mechanics) versus classical physical variables, which are differential functions of real variables. Heisenberg formally retained the equations of classical physics. (QED) In the case of quantum electrodynamics, these were Dirac’s spinors and multi-component wave functions, which, jointly, entail more complex operator variables, and a more complex structure of the corresponding Hilbert spaces, again, over complex numbers (apparently equally unavoidable in quantum field theory). In contrast to Heisenberg, Dirac also introduced a new equation, Dirac’s equation, formally different from Schrödinger’s equation, which, in accordance with the mathematical correspondence principle, is a far non-relativistic limit of Dirac’s theory, via Pauli’s theory, the immediate non-relativistic limit of Dirac’s theory. (3) A probabilistically predictive character of the theory. This change in mathematical variables was accompanied by a fundamental change in physics: the variables and equations of quantum mechanics and quantum electrodynamics no longer described, even by way of idealization, the properties and behavior of quantum objects themselves, in the way classical physics or relativity do for classical objects. Instead, the formalism only predicts the outcomes of events, in general probabilistically (even in the case of individual events), and of statistical correlations between some of these events, thus establishing the new type of relationships between mathematics and physics. This last feature is, arguably, the most controversial feature of quantum theory, from quantum mechanics to quantum field theory, a feature famously unacceptable to Einstein. Nevertheless, the probabilistic character of quantum theory is in accordance with the observable experimental data. For, identically prepared quantum experiments (in terms of the condition of the apparatuses involved), in general lead to different recordings of their outcomes, which makes predicting these outcomes, unavoidably probabilistic, although, again, certain multiplicities of quantum events also exhibit statistical correlations (not found in classical physics). In some respects, quantum phenomena are more remarkable for these correlations than for the irreducible randomness of individual quantum events. Perhaps the greatest of many enigmas of quantum physics is how random individual events combine into (statistically) ordered multiplicities under certain conditions, such as, most famously, those of the EPR (Einstein–Podolsky–Rosen) type experiments, considered in